{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "fc45c8e5",
   "metadata": {},
   "source": [
    "### 2.5.1 一个简单的例子"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "e0818cca",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([0., 1., 2., 3.])"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import torch\n",
    "x = torch.arange(4.0)\n",
    "x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "f906fb20",
   "metadata": {},
   "outputs": [],
   "source": [
    "x.requires_grad_(True) #等价于x = torch.arange(4.0, requires_grad = True)\n",
    "x.grad #默认值是None"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "58670b74",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor(28., grad_fn=<MulBackward0>)"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y = 2 * torch.dot(x,x)\n",
    "y"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b5c64990",
   "metadata": {},
   "source": [
    "调用反向传播函数来自动计算y关于x每个分量的梯度"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "12dcab36",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([ 0.,  4.,  8., 12.])"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y.backward()\n",
    "x.grad"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "a13a3265",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([True, True, True, True])"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x.grad == 4 * x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "cc9f7a8e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([1., 1., 1., 1.])"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#在默认情况下，pytorch会累积梯度，我们需要清除之前的值\n",
    "x.grad.zero_()\n",
    "y = x.sum()\n",
    "y.backward()\n",
    "x.grad"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0a5c22a3",
   "metadata": {},
   "source": [
    "### 2.5.2 非标量变量的反向传播"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7d60b976",
   "metadata": {},
   "source": [
    "我们调用向量的反向计算时，我们通常会试图计算一批训练样本中每个组成部分的损失函数的导数。这里我们的目的不是计算微分矩阵，而是单独计算批量中\n",
    "每个样本的偏导数之和。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "23096dd3",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([0., 2., 4., 6.])"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#对非标量调用backward需要传入一个gradient参数，该参数指定微分函数关于self的梯度\n",
    "#在我们的例子中，我们只想求偏导数的和，所以传递一个1的梯度是合适的\n",
    "x.grad.zero_()\n",
    "y = x * x\n",
    "#等价于y.backward(torch.ones(len(x)))\n",
    "y.sum().backward()\n",
    "x.grad"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "05a9a43c",
   "metadata": {},
   "source": [
    "### 2.5.3 分离计算"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "09c3d45e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([True, True, True, True])"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x.grad.zero_()\n",
    "y = x*x\n",
    "u = y.detach()\n",
    "z = u * x\n",
    "z.sum().backward()\n",
    "x.grad == u"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6083f3b0",
   "metadata": {},
   "source": [
    "### 2.5.4 Python控制流的梯度计算"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "57549307",
   "metadata": {},
   "source": [
    "即使构建函数的计算图需要通过python控制流，我们仍然可以计算得到的变量的梯度"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "75782632",
   "metadata": {},
   "source": [
    "### 2.5.5 小结"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "31db64f8",
   "metadata": {},
   "source": [
    "深度学习框架可以自动计算导数，我们首先讲梯度附加到想要对其计算偏导数的变量上，然后我们记录目标值的计算，执行它的反向传播函数，并访问得到\n",
    "梯度。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d445aa62",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
